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Abstract 

Leaders  at  the  United  States  Coast  Guard’s  Aircraft  Repair  and  Service  Center 
(ARSC)  in  Elizabeth  City,  North  Carolina  recently  formalized  their  planning  and  analysis 
functions  by  adding  a  dedicated  branch  to  their  command  structure.  The  Planning  and 
Analysis  Branch  intends  to  apply  computer  modeling  and  simulation  to  study  the  impact 
of  process  changes  to  the  various  Programmed  Depot  Maintenance  (PDM)  lines.  This 
research  considers  the  applicability  of  this  type  of  modeling  and  simulation,  using 
ARENA  to  study  the  current  HH-60J  PDM  process.  The  contribution  of  this  research  is  a 
methodology  specific  to  ARSC  needs,  an  analysis  of  methodology  based  on  a  discrete 
event  simulation  model  of  PDM  lines,  and  a  specific  case  study  demonstrating  the 
methodologies.  The  response  variable  of  interest  is  average  PDM  process  time  as  a 
function  of  either  in-sourcing  or  out- sourcing  labor  for  a  major  process  step.  The 
research  includes  development  and  evaluation  of  a  macro- level  process  model  using 
ARENA  5.0. 
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A  SIMULATION-BASED  ANALYSIS 
OF  THE  IMPACT  OF  IN-SOURCING  A  MAJOR  PROCESS  ELEMENT 
ON  THE  COAST  GUARD  HH-60J  DEPOT  MAINTENANCE  PROCESS 


I.  Introduction 

Background 

In  an  effort  to  reduce  costs  and  improve  service  to  its  customers,  leaders  at  the 
U.S.  Coast  Guard’s  Aircraft  and  Repair  Service  Center  (ARSC)  in  Elizabeth  City,  North 
Carolina  have  recently  added  a  Planning  and  Analysis  Branch.  Traditionally,  process 
analysis  and  optimization  were  performed  on  an  as-needed  basis  either  by  managers  and 
supervisors  from  a  variety  of  in-house  work  units  or  by  an  outside  contractor.  The 
standup  of  the  Planning  and  Analysis  Branch  signals  a  high-level  commitment  by  ARSC 
to  bring  state  of  the  art  modeling  and  analysis  techniques  and  tools  into  greater  use  for  in- 
house  decision  making. 

The  Planning  and  Analysis  Branch  is  currently  considering  a  change  to  the 
Programmed  Depot  Maintenance  (PDM)  process  for  the  Coast  Guard’s  HH-60J  search 
and  rescue  helicopter.  Vibration- induced  cracks  in  one  or  more  of  the  HH-60J’s  main 
beams,  essentially  the  airframe’s  structural  skeleton,  frequently  require  replacement  of 
the  entire  main  beam.  This  has  historically  been  done  by  on  site  at  ARSC  by  an  outside 
contractor  on  an  as- needed  basis.  ARSC  is  considering  purchasing  the  necessary  tools 
and  jigs  and  hiring  additional  workers  to  perform  these  repairs  organically,  without  the 
contractor. 
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The  equipment  costs  associated  with  adding  beam  replacement  capability  are 
relatively  simple  to  define.  However,  the  true  benefit  of  adding  workers  requires 
considering  of  the  value  of  their  contributions  to  the  entire  PDM  process.  This  study 
seeks  to  better  understand  how  each  of  the  two  beam  replacement  strategies,  organic  or 
contracted,  might  affect  the  overall  performance  of  the  PDM  process. 

Programmed  Depot  Maintenance 

The  Coast  Guard’s  aircraft  maintenance  program  consists  of  three  major  types  of 
maintenance  actions.  Unit- level  maintenance,  referred  to  as  organizational- level 
maintenance,  involves  routine  repairs  and  preventative  maintenance  performed  by  Coast 
Guard  technicians  at  a  Coast  Guard  Air  Station.  Some  unit-  level  repairs  require 
extensive  back- shop  work  and  may  involve  special  tools  and  expertise  that  are  usually 
associated  with  intermediate- level  maintenance. 

The  second  type  of  maintenance  action  is  major  modification,  commonly  referred 
to  as  a  mod.  Mods  include  system  upgrades  or  large-scale  repairs  to  add  capabilities  or 
correct  deficiencies  in  an  aircraft.  Mods  are  sometimes  performed  by  unit  personnel,  but 
are  more  commonly  performed  by  special  maintenance  teams  either  at  ARSC  or  at  the 
Air  Stations. 

The  third  type  of  maintenance  action,  programmed  depot  maintenance  (PDM),  is 
the  focus  of  this  research.  PDM  is  a  thorough  overhaul  of  the  aircraft,  generally 
consisting  of  the  removal  of  all  major  components,  inspection  of  the  airframe  and 
removed  components,  complete  reconditioning  of  aging  parts,  reassembly  and  repainting 
of  the  aircraft.  For  all  U.  S.  Coast  Guard  HH-60J  helicopters,  this  work  is  performed  at 
ARSC.  Aircraft  completing  PDM  are  restored  to  a  like-new  condition. 
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Programmed  Depot  Maintenance  Cycle 


PDM  schedules  are  planned  to  avert  aircraft  failures  associated  with  high 
accumulated  flight  times.  The  probability  of  failure  of  aircraft  systems  is  often  described 
by  a  bathtub  shaped  probability  distribution  (Figure  1).  Ebeling  describes  the  cycle  as  a 
piecewise  function  characterized  initially  by  bum-  in  failures,  then  by  random  mid-  life 
failures  and,  finally,  by  late- life  wearout  failures  (1997,  31).  The  high- incidence  failures 
early  in  the  cycle  can  be  identified  and  corrected  through  a  series  of  initial  test  flights. 
The  failure  rate  then  decreases  dramatically  and  tends  to  stabilize  until  the  aircraft 
accumulates  enough  flight  time  and/or  system  cycles  to  enter  the  region  of  increasing 
failures. 

▲ 

Probability  of 
Failure 


- ► 

Time 


Figure  1.  Reliability  Bathtub  Curve 
(Adapted  from  Ebeling,  1997) 

An  aircraft  PDM  avoids  the  steepest  part  of  the  curve  which  represents  the  aircraft 
age  at  which  failure  rates  increase  significantly.  The  PDM  concept  assumes  that  a 
properly  overhauled  aircraft  will  have  reliability  characteristics  similar  to  new  aircraft.  A 
PDM  interval  is  the  elapsed  time  between  start  of  consecutive  PDMs  for  a  given  aircraft. 
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By  managing  PDM  intervals,  ARSC  strives  to  continually  return  each  aircraft  to  like-new 


condition,  avoiding  increased  failure  rates  associated  with  older  aircraft.  For  the  HH-60J, 
the  target  PDM  interval  is  five  years.  This  equates  to  a  flow  rate  (work  in  process)  of  5 
aircraft  with  a  nominal  process  time  of  146  work  days,  based  on  an  assumed  245  work 
days  per  year  for  the  work  force.  A  significant  increase  in  process  time  would  require  an 
increase  in  flow  rate  in  order  to  meet  the  fleet- wide  target  PDM  interval.  This 
relationship  is  illustrated  by  Figure  2. 


PDM  Interval  vs.  Process  Time 


Figure  2.  Effect  of  change  in  process  time  on  WIP 

In  practice,  work  in  process  (WIP)  is  limited  by  available  floor  space  inside 

ARSC’s  PDM  hangar  and  by  operational  commitments.  The  Coast  Guard  operational 
concept  for  the  HH-60J  is  based  on  a  fleet  of  42  aircraft,  5  of  which  are  in  PDM  at  a 
given  time.  Figure  3  illustrates  the  resulting  relationship,  where  work  in  process  is  held 
constant  at  five  units  and  changes  in  PDM  cycle  times  affect  the  PDM  interval. 
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WIP  vs.  Total  Process  Time 


Figure  3.  Effect  of  Total  Process  Time  on  PDM  Interval 

As  these  models  illustrate,  a  reduction  in  PDM  cycle  time  benefits  ARSC 
either  by  reducing  requirements  for  WIP,  by  shortening  the  time  between  PDM  cycles,  or 
by  some  combination  of  the  two. 

Problem  Statement 

The  existing  beam  replacement  process  for  Coast  Guard  HH-60J  helicopter  relies 
on  an  outside  contractor  to  provide  labor,  special  tools  and  assembly  jigs.  ARSC  is 
considering  purchasing  the  necessary  equipment  and  hiring  additional  workers  to  perform 
beam  replacement  in-house  using  organic  resources.  An  increase  in  the  workforce  may 
provide  additional  benefits  that  extend  beyond  completion  of  required  beam 
replacements.  The  complexity  of  the  PDM  process  makes  it  difficult  to  correctly  predict 
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the  full  impact  of  additional  structures  workers.  ARSC  decision  makers  need  to  better 


understand  the  effects  of  beam  replacement  labor  resources  on  aircraft  process  time. 

Research  Questions 

This  research  is  designed  to  answer  the  following  questions: 

•  Is  an  ARENA  model  the  appropriate  tool  to  conduct  this  research  study? 

•  How  well  does  the  ARENA  model  represent  the  effects  of  changes  in  labor 
resources  for  ARSC’s  HH-60J  PDM  line? 

•  Will  an  increase  in  structures  shop  labor  likely  reduce  PDM  process  time  as 
compared  to  having  beam  replacement  work  done  by  a  dedicated  crew  of  contract 
workers? 

•  What  improvements  can  be  made  to  the  model  or  data  sources  to  develop  a  better 
ARENA  model  for  future  research? 

Summary  of  Current  Knowledge 

The  overall  PDM  process  is  well  defined.  Historical  process  times  for  HH-60J 
PDM  are  available  as  well  as  a  complete  history  of  aircraft  arrival  and  completion  times. 
The  flow  chart  in  Figure  4  depicts  an  aggregate  view  of  the  HH-60J  overhaul  process. 
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Figure  4.  Basic  HH-60J  PDM  Process 

Within  ARSC’s  databases,  labor  hours  are  charged  to  each  PDM  aircraft  by  work 
center  and  type  of  work  performed.  The  current  categorization  of  labor  expenses  is 
designed  to  meet  ARSC’s  internal  cost  accounting  practices.  The  data  that  align  best 
with  the  requirements  for  this  research  are  macro  process  times  and  labor  hours.  These 
labor  hour  data  are  difficult  to  attribute  to  detailed  work  tasks.  ARSC’s  emphasis  on 
macro- level  hourly  charges  mean  that  any  actual  data  is  more  appropriate  for  an 
aggregate- level  model  versus  a  detailed,  task-oriented  model. 

According  to  ARSC  analysts,  computer  modeling  is  a  new  organic  analysis  tool 
for  examining  the  HH-60J  PDM  line.  The  modeling,  methodology,  simulation  analysis 
techniques,  and  beam-replacement  focused  case  study  in  this  research  will  be  a  basis  for 
future  modeling  and  analysis  efforts  within  ARSC. 
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Assumptions 


The  simulation  model  that  serves  as  the  primary  research  tool  for  this  thesis 
depicts  a  macro  view  of  the  PDM  process.  Within  each  macro  process  step,  there  are 
actually  detailed  sub -processes  that  consume  resources  of  time,  labor  and  materials  at 
varying  rates.  A  macro- level  simulation  model  assumes  that  each  discrete  process  step 
can  be  aggregated  into  a  high-level  process  step  for  modeling  simplicity.  Interviews  with 
line  managers  at  ARSC  suggest  that  this  approach  is  appropriate  because  line  supervisors 
are  empowered  to  adjust  the  sequence  of  specific  tasks  within  an  aggregate  process  step 
to  keep  the  overall  PDM  project  on  schedule.  Without  aggregating  tasks,  accurately 
depicting  all  possible  task  sequence  combinations  would  be  difficult,  and  impractical,  to 
build,  particularly  since  we  want  to  model  the  entire  PDM  process,  not  just  some  piece  of 
the  process. 

Resource  tracking  modules  within  the  computer  model  assume  aggregation  of 
labor  within  each  work  center.  This  means  that  any  task  which  requires  labor  from  a 
given  work  center  can  be  performed  by  any  craftsman  in  that  work  center.  It  also  means 
that  modeling  of  labor  hour  divisions  may  not  exactly  match  actual  task  requirements. 
That  is,  a  task  scheduled  to  be  performed  by  one  person  for  two  hours  may  be  executed  in 
the  simulation  by  assigning  two  persons  for  one  hour  each.  In  some  cases  this  may  give 
an  optimistic  view  of  system  capabilities.  In  practice,  the  manner  in  which  craftsmen  are 
routinely  redirected  to  assist  fellow  workers  to  keep  project  work  flowing  suggests  that 
this  approach  is  appropriate. 
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Scope 


This  study  focuses  on  the  likely  outcome  of  two  different  ARSC  staffing  policies 
for  conducting  beam  replacement  step  during  PDM.  The  primary  measure  of  interest  is 
total  process  time  for  any  aircraft  completing  PDM.  However,  a  major  objective  of  the 
research  is  also  to  explore  how  well  simulation  methodology  in  general  and  our  specific 
model  serves  to  provide  insight  into  the  PDM  process. 

Thesis  Overview 

A  literature  review  of  simulation  theory  and  modeling  experiments  provides  a 
precedence  for  computer  modeling  of  the  PDM  process.  A  recent  AFIT  project  by 
Shyong  (2002)  studied  the  effects  of  inventory  levels  on  cost  and  process  times  for  depot 
level  repair  of  a  turbine  engine  component.  Shyong  used  the  ARENA  modeling  tool,  as 
does  the  ARSC  analysis  branch.  Thus,  the  decision  to  use  ARENA  for  this  project  was 
the  practical  choice. 

The  simulation  model  built  is  a  macro  model  of  the  PDM  line  for  H-60J 
airframes.  This  model  views  the  process  as  a  single  overhaul  line  that  receives 
helicopters  from  Coast  Guard  units,  performs  the  required  overhaul  steps,  and  outputs 
rebuilt  aircraft.  Some  aircraft  will  require  replacement  of  one  or  more  main  beams  as  a 
part  of  the  PDM  process.  The  simulation  model  accommodates  this  process  difference  by 
routing  individual  entities  according  to  work  required.  It  also  collects  time  statistics  for 
each  entity  that  passes  through  the  process  for  later  analysis. 

Simulation  theory  requires  two  reviews  of  a  model:  verification  and  validation. 
Verification  is  a  review  of  the  computer  code  and  process  structure  to  confirm  that  the 
model  is  constructed  correctly  with  respect  to  the  actual  structure  of  the  process  being 
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modeled.  It  also  confirms  that  there  are  no  errors  in  the  basic  programming  flow,  syntax, 
or  data  entry.  Validation  is  a  common- sense  review  of  the  model  to  confirm  that  outputs 
produced  by  the  simulation  match  the  real  world  (Kelton,  et  al,  2002:  43). 

In  the  verification  phase,  AFIT  experts  review  the  ARENA  model  code  for  errors. 
The  logical  structure  of  the  macro  model  is  reviewed  by  the  ARSC  sponsor  to  ensure  that 
it  matches  the  flow  of  the  real-world  system. 

Validation  involves  model  mns  using  process  times  and  induction  intervals  based 
on  historical  data.  Because  actual  labor  resource  requirements  for  each  sub -process  step 
are  not  clearly  defined  in  the  data,  it  is  impossible  to  completely  cross-check  the  behavior 
of  all  model  elements  against  the  real-world  system.  Validation  of  this  model  relies  on  a 
judgment  by  researchers  that  its  behavior  is  consistent  with  what  is  expected  from  the 
real-world  system.  Because  the  computer  model  structure  parallels  the  verified  macro 
model  logic,  validation  of  the  computer  model  also  includes  confirming  that  the  ARENA 
model  logic  matches  the  macro  model  logic. 

A  two  scenario  experiment  is  mn  with  the  verified  and  validated  model  to  predict 
the  effects  of  two  different  staffing  policies  for  the  beam  replacement  process  step. 
Scenario  one  assigns  the  required  labor  to  a  dedicated  team  of  five  contract  workers  who 
work  on  beam  replacement  exclusively.  Scenario  two  increases  the  ARSC  structures 
shop  workforce  by  five  workers,  allowing  them  to  work  on  other  PDM  tasks  when  not 
engaged  in  beam  replacement  work.  The  average  process  times  for  PDM  aircraft  in  each 
scenario  are  compared  to  determine  whether  scenario  two,  organic  beam  replacement, 
decreases  average  process  time. 
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Analysis  of  results  includes  evaluation  of  the  results  of  the  process  time  study  as 
well  as  an  assessment  of  the  ARENA  model  as  a  suitable  research  tool  for  additional 
studies  of  the  PDM  process. 
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II.  Literature  Review 


Introduction 

This  literature  review  briefly  summarizes  some  of  the  published  works  that 
support  the  basic  premises,  methodology,  and  conclusions  of  this  research.  It  begins  with 
background  information  on  Coast  Guard  HH-60J  helicopters.  It  then  defines  each  of  the 
two  measures  of  effectiveness:  time  and  schedule.  Next,  it  reviews  the  applicability  of 
various  scientific  approaches  to  evaluating  depot  maintenance  processes.  Finally,  it 
considers  the  applicability  of  the  selected  research  approach,  simulation,  and  defines  key 
terms  and  concepts  used  in  computer  simulation. 

Coast  Guard  HH-60J  Programmed  Depot  Maintenance 

The  United  States  Coast  Guard  operates  a  fleet  of  42  HH-60J  “Jayhawk” 
helicopters  located  at  twelve  Air  Stations  throughout  the  United  States.  These  aircraft  are 
stationed  at  coastal  units  and  are  used  for  a  variety  of  missions  including  search  and 
rescue,  law  enforcement  and  marine  environmental  safety.  Coast  Guard  personnel 
perform  day-to-day  maintenance  at  each  aircraft’s  base  of  operation,  forward  deployed 
site,  or  shipboard.  These  maintenance  actions  range  from  minor  inspections  to 
replacement  of  large  subassemblies  such  as  engines  or  transmissions.  Generally,  major 
maintenance  actions  and  system  upgrades  that  involve  extensive  disassembly  of  the 
airframe  or  excessive  labor  are  performed  at  ARSC  in  Elizabeth  City,  North  Carolina. 
ARSC  planners  use  a  master  PDM  schedule  to  sequence  the  complete  overhaul  of  each 
aircraft  in  the  fleet  based  on  each  aircraft’s  time  since  last  PDM.  Generally,  each 
airframe  returns  to  ARSC  every  five  years  for  PDM.  A  major  driver  of  this  five  year 
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PDM  interval  is  the  airframe  corrosion  that  develops  due  to  the  harsh  Coast  Guard 
operating  environment. 

The  scheduled  process  for  completing  PDM  requires  128  days.  This  schedule 
allows  for  repairs  and  upgrades  to  electrical,  mechanical,  avionics,  and  structural 
components  and  systems  as  well  as  all  normally  required  corrosion  control  and  paint 
work.  Most  of  the  work  is  performed  in-house  by  a  team  of  coast  guard  active  duty  and 
civilian  craftsmen. 

Measures  of  Effectiveness 

The  primary  measure  of  effectiveness  considered  in  this  research  is  process  time. 
Gilbride  (2002)  examined  the  PDM  outsourcing  decision  process  and  found  that  process 
time  is  valued  by  the  Coast  Guard  and  should  weigh  heavily  in  the  selection  of  the  best 
source  for  PDM  work.  This  is  consistent  with  the  expressed  priorities  of  ARSC,  the 
sponsor  of  this  research. 

Process  time  means  task  duration  and  is  related  to  output  rate  as  given  by  the 
following  formula: 

pt  =  -  (1) 

r 

where  pt  is  process  time,  and  r  is  desired  output  rate,  (adapted  from  Krajewski  and 
Ritzman,  2001:  470)  The  PDM  process  time  is  the  basis  for  the  overhaul  schedule,  and 
determines  the  length  of  time  a  given  aircraft  will  be  unavailable  for  operational  service. 
The  relationship  between  operational  availability  and  maintenance  cycle  time  is  described 
by  the  formula: 


,  MTBM 

A0  — 

MTBM  +  MDT 


(2) 
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where  MTBM  is  the  mean  time  between  maintenance  and  MDT  is  maintenance  down 


time  (Blanchard,  1998:127).  In  the  PDM  environment,  PDM  process  time  is  the  MDT 
used  to  calculate  operational  availability.  As  process  time  decreases,  MDT  decreases, 
and  operational  availability  increases. 

Theory  of  Constraints 

A  popular  approach  to  optimizing  performance  of  a  sequential  process  is  to 
identify  the  most  constrained  resource  and  focus  improvement  efforts  there.  The  term 
constrained  resource  suggests  something  in  limited  supply  such  as  raw  materials, 
machine  cycle  rate,  manpower,  etc.,  and  is  commonly  referred  to  as  a  bottleneck. 
Throughput  of  the  bottleneck  must  be  improved  in  order  to  improve  throughput  of  the 
entire  system.  (Goldratt,  1990:  5) 

Roser  et  al.  (2001:  949)  examined  various  approaches  to  identifying  bottlenecks 
in  a  process  and  concluded  that  doing  so  is  no  trivial  matter.  They  found  two  prominent 
techniques  for  analyzing  processes  for  bottlenecks:  measuring  wait  time  before  process 
steps,  and  calculating  overall  utilization  of  the  resource.  In  each  case,  the  resource 
generating  the  highest  value  is  the  bottleneck.  They  point  out  that  this  approach  is 
limited  in  that  it  concentrates  on  machine  utilization  and  virtually  ignores  the  effects  of 
other  elements  such  as  supply  and  demand,  and  human  workers.  They  offer  a  more 
robust  model  for  identifying  bottlenecks  by  considering  these  and  other  factors. 
Modeling  Approaches 

While  modeling  tools  for  decision  making  are  varied  and  many,  they  can  each  be 
categorized  as  either  prescriptive,  predictive,  or  descriptive.  Ragsdale  (2002:  8)  offers 


14 


two  criteria  for  evaluating  into  which  category  a  model  fits:  form  of  the  function  f(*),  and 


values  of  independent  variables: 


Table  1.  Modeling  Techniques 


(Ragsdale,  2002) 

Values  of 
Independent 

Modeling 

Category 

Prescriptive 

Models 

Form  of  f(*) 

known, 

well-defined 

Variables 

known  or  under 

decision-maker’s 

Control 

Techniques 

Linear  Programming, 
Networks,  Integer 
Programming,  Critical  Path 
Modeling  (CPM), 

Goal  Programming, 
Economic  Order  Quantity, 
Nonlinear  Programming 

Predictive 

Models 

unknown, 
ill- defined 

known  or  under 

decision-maker’s 

control 

Regression  Analysis, 

Time  Series  Analysis, 
Discriminant  Analysis 

Descriptive 

Models 

known, 

well-defined 

unknown  or 
uncertain 

Simulation,  Queuing, 
Program  Evaluation  and 
Review  Techniques  (PERT), 
Inventory  Models 

Applying  these  guidelines,  a  model  of  the  PDM  process  is  classified  as  a 
descriptive  model.  The  cause  and  effect  relationships  between  process  variables  and 
system  outcomes  are  well  known,  and  variability  in  process  times  makes  the  values  of 
independent  variables  uncertain.  As  we  note  from  Table  1,  simulation  is  an  appropriate 
tool  for  this  type  of  model. 
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Vashi  and  Bienstock  (1995:  197)  focus  specifically  on  logistics  applications  when 
describing  three  popular  modeling  approaches:  optimization,  heuristics,  and  simulation. 
They  analyze  pros  and  cons  of  each  and  encourage  the  potential  application  of  multiple 
approaches  to  the  same  problem. 

The  first  approach,  optimization,  prescribes  a  best,  or  optimal,  combination  of 
values  for  variables  over  a  given  range  to  maximize  the  objective  value  of  some 
mathematical  programming  model.  For  a  given  range  of  values,  this  should  prove  to  be 
the  most  effective  approach.  However,  when  the  exact  range  of  values  for  variables  is 
unknown,  and  the  model  produces  some  optimal  solution  for  that  solution  space,  that 
solution  may  not  necessarily  be  the  optimal  solution  for  the  problem  at  hand.  A  second 
limitation  involves  large  solution  spaces  that  can  require  unreasonable  computer 
processing  times  to  produce  a  solution.  In  these  cases,  the  computer  program  will  stop  its 
search  according  to  a  predetermined  heuristic.  This  limiting  approach  carries  no 
guarantee  of  finding  the  true  optimal  solution  but  may  be  the  only  viable  approach  when 
applying  optimization  to  complex  problems  such  as  the  PDM  process. 

The  second  approach,  heuristics,  applies  a  “rule  of  thumb”  to  the  decision 
process.  Often  these  heuristics  are  based  on  user  experience  and  can  often  lead  the 
decision  maker  to  a  reasonably  good  solution.  There  is,  however,  no  guarantee  that  this 
solution  is  the  optimal  choice.  The  approach  is  also  somewhat  limited  because  it  is  based 
on  previous  experience.  It  is  unlikely  that  an  innovative  new  solution  will  be  found 
unless  the  heuristic  model  itself  provides  some  mechanism  for  generating  innovations. 

The  third  approach,  simulation,  was  selected  for  use  in  this  research.  Simulation 
addresses  the  special  requirements  of  a  complex  processes,  not  only  by  allowing  for 
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variability,  but  also  by  facilitating  stochastic  analysis.  The  PDM  process  is  affected  by 
changes  in  process  times,  lead  times,  arrival  times,  inventory  levels,  etc.  Computer 
simulation  allows  researchers  to  model  stochastic  processes  and  to  analyze  the  effects  of 
various  policies,  not  only  on  the  objective  function,  but  also  on  each  intermediate 
variable,  and  derive  probability  distributions  for  a  range  of  results  versus  a  single 
predicted  output  value. 

Related  Logistics  Research  in  Modeling  and  Simulation 

There  are  numerous  examples  of  successful  application  of  simulation  to  aspects  of 
the  depot  maintenance  process.  Shyong  (2002)  evaluated  the  effects  of  various  spare 
parts  levels  and  queuing  policies  on  process  time  and  cost  for  the  overhaul  of  the  F101 
LPT  rotor  at  Tinker  Air  Force  Base.  His  detailed  model  of  both  front-  and  back-shop 
activities  met  verification  and  validation  criteria  and  identified  potential  savings  in  both 
time  and  cost  for  this  step  in  the  overhaul  process.  More  importantly,  his  research 
demonstrated  the  value  of  simulation  in  evaluating  cost  and  time  improvement 
opportunities  in  other  engine  overhaul  sub -processes.  (Shyong,  2002) 

Mooney  (1997)  studied  turn  around  time  by  experimenting  with  a  model  of  a 
single  critical  hydraulic  control  part.  His  model  predicted  process  times  for  repair  of  this 
component  at  the  Naval  Aviation  Depot.  By  modeling  the  effects  of  various  changes  in 
the  process  flow,  he  was  able  to  identify  process  improvements  to  generate  savings  in 
cost  and  process  time.  (Mooney,  1997) 

Schuppe  et  al  (1993)  modeled  the  addition  of  two  major  process  tasks  to  the  C- 
141  airplane  PDM  process.  They  applied  simulation  to  predict  the  effects  of  the  increase 
in  work  tasks  on  existing  PDM  schedules  and  resources  (people,  hangar  space,  test 
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equipment,  tooling,  and  money).  The  simulation  predicted  that  the  new  process  steps 
would  result  in  a  shortfall  of  production  needed  to  meet  customer  requirements.  The 
team  identified  a  few  sub -processes  as  bottlenecks  requiring  additional  attention  as  likely 
sources  of  process  improvement. 

Computer  Simulation  Concepts  and  Methods 

Kelton  et  al  (2002:  8)  report  that  simulation  leads  all  other  operations  research 
tools  in  popularity.  The  almost  ubiquitous  presence  of  powerful  desktop  computers  in 
business  offices  places  the  necessary  computing  power  for  simulation  studies  within 
reach  of  most  managers. 

Early  non-computerized  simulations  and  modem  analytic  models  were  limited  in 
their  complexity  by  the  user’s  ability  to  process  data.  This  limitation  was  usually 
addressed  by  generalizing  the  performance  of  elements  of  the  model  and  by  making 
assumptions  about  interactions  of  entities  and  values  of  variables.  When  dealing  with 
complex  systems  such  as  a  PDM  line,  these  simplifications  can  render  the  model 
ineffective.  Computer  models  can  be  built  as  complex  or  as  simple  as  necessary  to 
provide  the  necessary  level  of  detail  for  the  system  elements  being  studied. 

Models  of  large-scale  systems  quickly  become  very  complex,  taxing 
programming  resources  and  computer  processing  power.  Increases  in  desktop  computing 
power  and  the  availability  of  high-level  simulation  languages  now  allow  researchers  to 
experiment  with  large,  detailed  models  that  were  previously  impossible  to  work  with. 
(Law  and  Kelton,  2000:  2) 
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Model  Verification  and  Validation 


Verification  of  a  model  is  the  process  that  confirms  the  model  faithfully 
represents  the  conceptual  model  (Kelton,  et  al  2002:  42).  This  involves  reviewing  the 
logic  with  system  experts  and  considering  the  effects  of  possible  inputs  and  interaction 
within  model  elements.  A  properly  verified  model  accurately  represents  the  system 
concept  that,  in  turn,  represents  the  elements  of  the  real  world  system  under  study. 

Kelton,  et  al  (2002:  43)  associate  validation  with  a  comparison  of  simulation 
results  against  observations  of  the  actual  process.  The  range  of  results  from  a  valid 
simulation  should  encompass  results  from  the  real-world  process.  However,  many 
simulations  model  rely  on  abstraction  of  the  model  for  simplicity  or  are  designed  to  study 
the  results  of  scenarios  for  which  there  are  no  real-world  examples.  Law  and  Kelton 
(2000:  86)  address  the  case  where  there  is  no  existing  system  for  comparison.  They 
recommend  having  analysts  and  experts  review  the  model  for  correctness  and 
reasonableness  of  the  model  outputs. 
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III.  Methodology 


Introduction 

This  chapter  discusses  the  research  methodology.  It  explains  the  basis  for 
selecting  simulation  modeling  as  a  research  vehicle  and  describes  the  process  used  to 
define  and  develop  the  final  model.  It  then  describes  the  specific  functions  of  major 
model  elements,  model  logic  structure,  and  methods  used  to  collect  and  analyze  output 
data.  It  also  reviews  the  verification  and  validation  process  that  was  employed. 

Selection  of  Computer  Simulation  as  the  Research  Vehicle 

A  variety  of  tools  are  available  to  study  the  behavior  of  a  system.  Law  and 
Kelton  (2002)  offer  a  discussion  of  these  options  and  an  illustration  (adapted  as  figure  5) 
that  shows  how  these  methods  relate  to  each  other  and  the  type  of  system  to  be  studied. 
One  decision  that  a  researcher  makes  is  whether  to  experiment  with  the  actual  system  or 
to  experiment  with  a  model  of  the  system.  For  a  complex  system  such  as  the  Coast 
Guard  helicopter  PDM  line,  experiments  with  the  actual  system  are  impractical  due  to  the 
long  cycle  times.  Actual  cycle  times  for  the  process  vary,  but  considering  a  nominal  146 
day  cycle  time,  with  five  units  in  the  PDM  process,  it  would  take  over  a  year  to  collect 
data  on  10  aircraft  PDM  cycles.  For  this  reason,  experimentation  with  a  model  of  the 
system  was  selected. 
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System 


Figure  5.  Ways  to  study  a  system 

Adapted  from  Law  &  Kelton  (2002:  4) 


Among  modeling  options  are  physical  models  and  mathematical  models. 
Physical  models  generally  employ  some  type  of  scale  model  to  examine  the  effects  of 
changes  in  the  physical  system  components.  In  this  study,  the  variables  of  interest  focus 
more  on  event  scheduling  than  on  the  physical  plant  layout.  For  this  reason,  a 
mathematical  model  is  best  suited  to  representing  system  behavior. 

Within  the  category  of  mathematical  models  are  analytical  solutions  and 
simulation.  The  former  approach  requires  a  complete  understanding  of  the  exact 
relationships  between  processes  and  functions  that  comprise  the  system.  It  also  requires 
that  the  system  be  simple  enough  to  allow  researchers  to  develop  accurate  mathematical 
formulae  to  describe  the  interaction  of  system  elements  and  calculate  the  value  of  all 
output  variables  of  interest.  Stochastic  process  techniques  offer  a  viable  option. 
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However,  complexity  of  the  PDM  process  and  the  complex  interactions  between  system 
elements  over  time  seemed  to  indicate  that  simulation  is  the  best  approach  to  system 
modeling. 

Conducting  a  Simulation  Study 

Researchers  have  applied  slightly  different  procedures  for  developing  and 
conducting  simulation  studies.  Shyong  (2002:29)  based  his  design  and  analysis  process 
on  the  models  presented  by  both  Montgomery  (1991:9)  and  Altiok  and  Melamed  (2001: 
6),  resulting  in  a  proces  that  is  uniquely  adapted  to  simulation  of  the  details  of  a  specific 
element  of  the  PDM  process,  specifically  the  repair  of  the  low  pressure  turbine 
component  of  the  F101  turbine  engine.  Law  and  Kelt  on  (2002:84)  offer  a  model  which 
follows  the  same  general  flow  but  which  includes  a  feedback  path  that  suggests 
reviewing  previous  process  steps  after  verification  and  validation.  This  research  follows 
Law  and  Kelton’s  model  (figure  6). 
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Figure  6.  Steps  in  a  Simulation  Study 
(Law  and  Kelton:  2000:84) 

Formulating  the  Problem  and  Planning  the  Study 

The  objective  of  this  simulation  study  is  to  consider  the  possible  effects  of  beam 
replacement  staffing  and  scheduling  policies  on  the  overall  process  flow  time  for  HH-60J 
PDM.  The  basic  process  model  will  be  modified  to  create  two  models,  each  representing 
a  staffing  strategy. 
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The  first  model,  scenario  one,  simulates  the  current  policy  of  bringing  in  a  team  of 
contracted  technicians  who  work  exclusively  on  beam  replacement  for  those  aircraft 
requiring  this  work.  In  ARENA,  the  resource  for  this  work  is  Beam  Contractor. 

The  second  model,  scenario  two,  adds  to  the  in-house  team  of  structures 
technicians  who  can  work  on  any  process  tasks  requiring  structures  workers.  Beam 
replacement  work  takes  priority  over  other  structures  tasks  in  this  scenario. 

Collecting  Data  and  Defining  a  Model 

Line  supervisors  for  the  HH-60J  PDM  line  provided  detailed  maintenance 
schedules  for  their  respective  processes.  These  schedules  were  based  on  point  estimates 
of  process  times  for  each  major  task.  From  these  master  schedules,  an  equivalent  logic 
flow  diagram  was  generated  to  define  the  predecessor  relationships  between  concurrent 
tasks.  Some  sequential  tasks  that  involved  a  single  work  shop  were  combined  to  simplify 
the  model  logic.  Similarly,  some  tasks  were  broken  into  sub-tasks,  designated  as  initial 
and  final,  to  allow  partial  completion  of  one  task  to  function  as  a  predecessor  for  another 
task.  The  resulting  macro  model  is  included  as  Figure  7.  Supporting  sub- models  of  this 
macro  are  included  in  Appendix  A. 
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Figure  7.  HH-60J  PDM  Main  Model  Logic  Flow  Diagram 

For  modeling  purposes,  the  PDM  process  is  viewed  as  a  series  of  sequential  task 
groups.  The  model  assumes  that  workers  complete  the  induction  phase  first,  then  move 
on  to  disassembly,  strip  and  clean,  repair,  and  so  on.  Within  a  major  task  group,  tasks 
can  be  completed  sequentially,  simultaneously,  or  both.  These  task  groups  are  modeled 
in  ARENA  as  sub- models.  Figure  8  shows  the  tasks  and  logic  that  comprise  the 
Disassembly  Sub-Model. 
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Figure  8.  HH-60J  PDM  Disassembly  Sub-Model  Logic  Flow  Diagram 

For  the  example  in  figure  8,  task  DA2  (Remove  Avi  Rack  and  Components) 
commences  after  completion  of  task  DAI  (Remove  Antennas,  RTs,  Instruments).  Tasks 
DAI,  DEI,  DM1,  DM4  and  DM6  may  be  performed  simultaneously  if  adequate  labor 
resources  are  available.  The  five-input  AND  symbol  indicates  that  all  five  predecessor 
tasks  must  be  completed  before  completing  this  sub -task  and  starting  the  next  step  in  the 
macro  PDM  process  (the  task  groups  in  Figure  7). 

When  converted  to  ARENA  logic,  the  same  sub -model  is  represented  by  the  logic 
flow  illustrated  in  Figure  9.  The  initial  branch  to  five  parallel  process  steps  is 
accomplished  by  using  the  duplicate  block  from  ARENA’S  Blocks  menu.  The  duplicate 
block  creates  four  duplicate  entities  plus  the  original.  Each  of  the  five  entities  then 
proceeds  through  the  logic  independently  until  being  combined  back  into  a  single  entity 
at  the  Batch  block.  Once  divided,  each  entity  represents  a  portion  of  the  work  to  be 
performed  in  the  Disassembly  Sub-Model.  The  Batch  block  performs  the  AND  function 
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by  blocking  entities  from  flow  back  to  the  main  model  until  all  required  sub -model  work 


is  complete. 


n  r 


Figure  9.  HH-60J  PDM  Disassembly  ARENA  Sub-Model 

The  arrival  of  aircraft  to  PDM  is  scheduled  based  on  availability  of  production 
resources.  In  actual  ARSC  schedulers  coordinate  with  Air  Stations  so  that  a  single 
aircrew  can  deliver  a  completed  aircraft  to  an  air  station  and  pick  up  the  next  unit  for 
induction.  This  capacity- driven  arrival  sequence  is  modeled  in  ARENA  as  two  Create 
Entity  blocks  and  a  Seize  block  (Figure  10). 


27 


Figure  10.  HH-60J  Entity  Creation  and  Release  Logic 

The  first  five  entities  are  generated  according  to  a  constant  inter- arrival  time 
schedule  to  evenly  load  the  PDM  production  line.  Subsequent  entities  are  generated  after 
the  fifth  entity  and  stored  in  the  Seize  Floor  Space  queue  until  the  resource  Floor  Spaces 
becomes  available.  By  limiting  Floor  Space  to  five,  this  block  limits  work  in  process  to 
five. 

Validating  the  Conceptual  Model 

ARSC  staff  reviewed  the  logical  model  for  logic  flow  and  predecessor 
relationships  between  task  elements.  Changes  to  the  model  were  incorporated  prior  to 
construction  of  the  ARENA  model. 

Constructing  and  Verifying  the  Computer  Model  and  Making  Pilot  Runs 

An  ARENA  model  of  each  process  line  was  developed  using  the  validated  logic 
flow  diagrams  and  process  times.  Sub -models  were  developed  independently  and  tested 
as  stand-alone  modules  to  verify  correct  operation  before  being  integrated  into  the  main 
model.  A  master  data  dictionary  was  used  to  verify  the  correct  assignment  and  use  of 
element  names  and  assignment  of  resources.  An  element-by-element  review  was 
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conducted  to  verify  that  each  ARENA  data  field  matches  the  corresponding  data 
dictionary  entry.  Proper  interaction  of  sub-models  was  checked  by  observing  ARENA- 
generated  animatio  n. 

The  labor  hour  data  that  are  currently  available  for  the  HH-60J  PDM  process  are 
not  categorized  by  individual  work  center  and  work  task.  Labor  hour  data  are  further 
obscured  by  the  combination  of  front  shop  and  back  shop  tasks  involved  in  the  PDM 
process.  Individual  workers  may  be  assigned  to  front  shop  activities,  back  shop 
activities,  or  both.  Our  model  only  focuses  on  front -shop  activities.  This  model  deals 
with  the  floating  labor  pool  by  assuming  that  line  supervisors  schedule  workers  with  a 
priority  on  front  shop  tasks  to  meet  production  schedules.  Fluctuations  in  front  shop 
process  times  will  result  in  fluctuations  in  available  back  shop  labor.  For  the  existing 
PDM  process,  it  is  assumed  that  manpower  is  staffed  to  allow  for  routine  fluctuations 
without  negatively  impacting  the  completion  of  back  shop  work. 

ARSC’s  current  policy  for  beam  replacement  is  to  bring  in  contracted  workers, 
just  for  the  duration  of  the  replacement,  then  to  release  the  workers.  An  alternative, 
adding  in-house  resources,  assumes  that  the  current  structures  shop  manning  level  has 
historically  met  production  requirements,  cannot  accommodate  the  additional  labor 
demands  of  beam  replacement. 

To  determine  the  appropriate  structures  shop  manpower  level,  a  response  study 
was  conducted  using  ARENA’S  Process  Analyzer.  Initial  manning  levels  were  suggested 
by  ARSC  analysts  and  served  as  the  started  point  for  the  response  study.  Based  on 
multiple  mns  of  30  repetitions  each,  manning  levels  were  determined  for  each  shop, 
producing  a  mean  process  time  of  131.45  days.  Structures  worker  manning  was  set  at  11, 
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a  level  that  provided  a  minimum  process  time  that  is  sensitive  to  a  decrease  in  number  of 
structures  workers.  This  simulates  a  staffing  level  where  the  addition  of  a  major  task 
such  as  beam  replacement  would  impact  manpower  available  for  other  tasks,  either  in  the 
front  shop  or  in  the  back  shop. 

The  beam  replacement  step  was  added  to  the  model  by  inserting  one  Decide  block 
and  one  Process  block  (Figure  1 1).  The  modeled  probability  of  replacing  a  beam  (76%) 
is  based  on  historical  data  provided  by  ARSC.  The  beam  replacement  block  is  placed  in 
series  with  the  repair  process  flow  to  model  a  25  day  increase  in  process  time  due  to 
beam  replacement. 


Figure  11.  HH-60J  PDM  Beam  Replacement  Logic 

Validating  the  Programmed  Model 

The  graphical  interface  in  ARENA  allowed  construction  of  a  model  that  very 
closely  resembles  the  logic  diagram  reviewed  by  ARSC.  ARSC  analysts  validated  the 
logic  for  beam  replacement  by  reviewing  probability  and  process  time  assumptions.  The 
remaining  ARENA  logic  was  verified  by  comparing  ARENA  with  the  corresponding 
logic  element  in  the  logic  model.  The  process  times  and  resources  were  then  compared  to 
those  recorded  in  the  data  dictionary  and  the  original  project  schedule  from  which  the 
data  dictionary  was  developed. 
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Designing  Experiments 

The  study  utilizes  two  experiments.  Each  experiment  focuses  on  one  of  two 
staffing  policies.  In  the  design  phase,  values  were  determined  for  each  of  four  parameters 
for  these  experiments:  initial  entity  arrival  interval,  warmup  time  for  simulation, 
duration  of  simulation,  and  number  of  simulation  repetitions. 

Entity  arrival  interval  only  applies  to  the  first  five  entities.  When  one  of  these 
entities  completes  the  PDM  process  it  triggers  the  start  of  a  release  of  a  new  entity  from 
the  queue.  Process  output  intervals  were  inspected  visually  for  balanced  inter-arrival 
times  and  total  process  times  using  an  ARENA  plot  of  process  time  versus  completion 
time  for  a  single  10,000  day  simulation  (Figure  12).  A  20  minute  arrival  interval 
produced  an  output  pattern  that  evenly  distributes  initial  inter- completion  times  and 
minimizes  bunching  of  entities  completing  the  process. 
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Figure  12.  HH-60J  PDM  Entity  Process  Output  Intervals 


Warm-up  time  was  similarly  determined  using  a  visual  inspection  of  the  plot  of 
entity  total  process  time  versus  entity  number  (Figure  13)  The  process  time  pattern 
becomes  stable  as  soon  as  the  process  is  fully  loaded  with  five  entities.  Based  on  this 
observation,  the  warm-up  time  was  set  to  150  hours  to  delay  data  collection  until  the 
process  is  fully  loaded.  This  inspection  method  for  determining  warm-  up  time  is  offered 
by  Kelton,  et  al  (2002:  288)  as  an  appropriate  technique  for  determining  when  steady 
state  appears  in  a  simulation. 
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Figure  13.  HH-60J  Entity  Process  Times 

The  duration  of  the  simulation  was  selected  based  on  available  computing  power. 
The  10,000  hour  simulation  used  for  the  previously  described  testing  requires 
approximately  15  seconds  of  computing  time.  This  allows  the  experiment  to  be  run 
within  the  time  domain  that  initial  testing  was  done. 

To  estimate  the  number  of  repetitions  required,  results  of  the  initial  30  repetition 
runs  were  used.  Scenario  1  produced  an  average  process  time  of  147.14  days  with  a 
sample  standard  deviation  of  0.83  days.  Scenario  2  produced  an  average  process  time  of 
145.92  days  with  a  sample  standard  deviation  of  1.45  days.  A  target  95%  confidence 
interval  of  0.5  days  (with  a  corresponding  0.25  day  half-  width)  was  selected  for  the  final 
production  runs.  Initial  estimates  of  scenario  standard  deviations  were  used  to  estimate 
replications  required  to  achieve  the  specified  precision. 
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Law  &  Kelton  (2000)  offer  a  formula  for  calculating  a  point  estimate  and 
confidence  interval  for  the  population  mean,  given  the  results  from  n  independent 
replications  of  a  simulation. 

Cl  =  Xbar(n)  ±  tn_u.a  SQRT  [s2(n)/n]  (3) 

The  results  from  scenario  1  produced  the  largest  sample  variance.  This  worst- 
case  scenario  was  used  as  the  basis  for  calculating  the  baseline  confidence  interval: 

Cl  =  145.92  ±  1.96  •  SQRT[2. 10/30]  (4) 

Cl  =  145.92  +  0.52  (5) 

This  confidence  interval  exceeds  the  target  interval  of  +  0.25  days.  The  original 
formula  provided  a  basis  for  solving  for  N,  number  of  repetitions  required,  based  on  the 
results  of  30  repetitions: 


N=[(1.96  •  1.45)/ 0.25]2 

(6) 

N=129.2 

(7) 

Based  on  this  result,  the  theoretical  minimum  number  of  repetitions  to  achieve  the 
desired  confidence  interval  of  +  0.25  days  is  130.  Law  &  Kelton  (2000:  513)  emphasize 
that  this  is  only  an  approximation  of  the  number  of  required  replications  and  that  this 
formula  is  offered  as  a  tool  to  manage  computing  resources  when  unnecessary 
replications  will  waste  limited  computing  resources.  The  actual  computing  time  for  30 
replications  of  this  simulation  is  under  ten  minutes.  Thus,  we  increased  the  number  of 
repetitions  to  140,  increasing  the  likelihood  that  the  simulation  will  achieve  the  target 
confidence  interval  for  average  process  time. 

The  use  of  common  random  numbers  was  considered  as  a  means  to  reduce 
variance  in  comparing  two  systems.  This  study  considers  two  modeled  alternatives  that 


34 


include  the  same  number  of  process  steps,  suggesting  that  it  might  be  a  good  candidate 
for  synchronized  random  number  allocation.  Kelton,  et  al  (2002:  484)  describe 
synchronization  as  a  method  of  inducing  correlation  between  two  or  more  models  so  that 
a  more  accurate  comparison  can  be  made  between  them.  As  entities  pass  through  the 
ARENA  model  of  the  PDM  process,  they  experience  delay  times  based  on  the  defined 
probability  distributions  for  these  times.  The  actual  value  drawn  from  each  distribution  is 
determined  by  the  random  number  data  stream.  With  66  process  blocks  in  the  PDM 
ARENA  model,  the  sequence  of  entity  flow  through  the  model  is  substantially  affected 
by  the  interaction  between  these  random  draws.  If  all  entities  passed  through  the  model 
in  the  same  sequence,  then  assignment  of  random  number  streams  to  each  process  block 
could  synchronize  the  random  numbers  used  in  two  models.  However,  the  decide  block 
that  redirects  some  entities  to  beam  replacement  also  provides  a  means  for  entities  to  pass 
each  other  in  the  process  and  for  any  specific  aircraft  to  follow  different  processes  within 
each  scenario.  This  disruption  of  entity  sequencing  means  that  a  direct  comparison  of 
average  process  times  by  entity  is  not  practical. 

Production  Runs 

Two  scenarios  were  selected.  Scenario  1  models  a  process  with  five  dedicated 
contract  workers  assigned  to  work  on  beam  replacement  exclusively.  Scenario  2  models 
a  process  with  five  structures  workers  assigned  to  work  primarily  on  beam  replacement, 
but  assigned  to  work  on  other  available  structures  tasks  when  not  actively  working  on 
beam  replacement.  Beam  replacement  work  takes  priority  over  other  tasks  for  these 
workers. 
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Each  scenario  was  run  for  140  replications  of  10,000  hours  each  with  statistics 
collected  on  average  process  time  for  each  replication. 
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IV.  Results 


Introduction 

This  chapter  presents  the  results  of  the  experiment.  It  describes  the  steps  followed 
in  testing  output  data  and  offers  conclusions  based  on  these  results. 

Hypothesis  and  Hypothesis  Testing 

The  experiment  involves  a  hypothesis  test  designed  to  examine  whether  staffing 
the  beam  replacement  process  step  with  general  structures  shop  labor  will  provide  an 
improvement  in  process  time,  as  compared  to  having  a  dedicated  team  of  beam 
replacement  contractors  perform  the  work.  Given  outputs  from  simulations  of  the  two 
alternatives,  the  test  is: 

Ho:  Pi-P2=0 
Ha:  Pi-P2?0 

Where:  Pi  =  Average  process  time  for  scenario  one  and 

P2  =  Average  process  time  for  scenario  two. 

The  null  indicates  no  difference  in  process  times  between  scenario  one  and 
scenario  two.  If  the  test  rejects  the  null  hypothesis,  then  the  alternate  hypothesis,  there  is 
a  difference  in  process  times  between  scenario  one  and  scenario  two,  is  presumed  to  be 
accepted.  If  the  hypothesis  test  fails  to  reject  the  null  hypothesis,  then  no  conclusion 
regarding  the  alternate  hypothesis  can  be  made,  other  than  our  analysis  failed  to  depict 
any  difference  in  process  time. 
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144.5 


Test  for  Normality  of  Outputs  Means 

The  Central  Limit  Theorem  states  that  the  distribution  of  means  of  independent, 
random  samples  from  a  population  follow  a  normal  distribution.  The  output  from  of 
scenarios  one  and  two  were  tested  for  normality  using  the  Fit  Distribution  function  in 
@Risk  .  Both  samples  sufficiently  follow  the  normal  distribution  (see  Table  3). 

Table  2.  Entity  Process  Time  Goodness  of  Fit  Test  Results 


Normal(1 47.1 7400,  0.80717) 

o.ey - v - V - V 


Normal(1 45.7666, 1.5347) 


145.846  148.502 

Scenario  1  Goodness  of  Fit  (Normal) 


5.0% 


143.24  148.29 

Scenario  2  Goodness  of  Fit  (Normal) 


Chi- Squared  Test  Value  6.157 


11.54 


P  Value 


0.9080 


.4831 


Rank 


1 
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152- 


Testing  for  Equal  Variances 


Outputs  from  scenarios  one  and  two  produced  variances  of  0.65  and  2.36, 
respectively.  Excel’s  two-sample  test  for  equal  variances  was  employed  to  evaluate 
whether  there  is  any  statistical  difference  in  output  variance  between  the  two  scenarios. 
The  resulting  output  is  shown  in  Figure  4: 

Table  3.  F-Test  Two-Sample  for  Equal  Variances. 


Scenario  1 

Scenario  2 

Mean 

147.18 

145.77 

Variance 

0.65 

2.35 

Observations 

140 

140 

df 

139 

139 

F 

3.61 

P(F=f)  one-tail 

1.2  IE- 13 

F  Critical  one-tail 

1.32 

The  F  statistic  for  this  test  exceeds  the  F  critical  value,  supporting  a  conclusion  of 
unequal  variances  of  the  scenario  outputs. 

Hypothesis  Testing 

The  paired  t-test  was  employed  due  to  the  unequal  variances  of  the  scenario 
output.  This  test  produces  a  confidence  interval  for  the  difference  of  means.  If  the 
confidence  interval  contains  zero,  then  the  test  fails  to  reject  the  null.  The  added  value  of 
this  test  is  that  it  provides  a  description  of  the  difference  of  the  means,  which  can  be  used 
to  describe  the  difference  in  outputs  between  the  two  scenarios. 
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The  paired-T  test  does  not  require  equal  variances  but  it  does  require  equal 


sample  sizes.  It  also  accommodates  positive  correlation  between  the  two  samples. 
Although  the  experiment  was  not  specifically  designed  with  synchronized  common 
random  numbers,  ARENA’S  random  number  assignment  process  will  likely  result  in  a 
high  degree  of  correlation.  Law  and  Kelton  (2000:560)  note  that  many  simulation 
packages  allocate  random  numbers  in  a  manner  that  requires  specific  action  by 
programmers  to  defeat  the  high  degree  of  commonality  and  synchronization  between 
scenarios. 

Excel  was  used  to  conduct  the  paired-T  test  with  the  following  results  (Table  5): 

Table  4:  Paired-T  Test  for  Difference  of  Means 


Mean 

1.41 

Standard  Deviation 

1.77 

Standard  Error 

0.15 

95%  Confidence  Level 

0.29 

With  a  confidence  interval  that  does  not  contain  0,  the  null  hypothesis  is  rejected, 
supporting  the  conclusion  that  the  difference  in  mean  outputs  between  scenarios  one  and 
two  is  statistically  significant.  The  confidence  interval  also  describes  the  expected  range 
for  difference  in  mean  process  times  between  the  two  scenarios  as  1.12  to  1.70  days. 
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V.  Conclusions 


Introduction 

This  research  employs  modeling  and  simulation  to  predict  the  impact  of  in¬ 
sourcing  the  replacement  of  main  beams  by  ARSC’s  PDM  line.  The  primary  measure  of 
effectiveness  for  this  study  is  average  process  time. 

Conclusions 

Results  of  the  literature  review  summarized  in  Chapter  II  and  the  experiment 
described  in  Chapters  III  and  IV  support  answers  to  the  four  research  questions  presented 
in  Chapter  I: 

1.  Is  an  ARENA  model  the  appropriate  tool  to  conduct  this  research  study? 

A  review  of  current  literature  supports  the  use  of  computer  simulation  for  this 
type  of  study.  Much  of  the  previously  published  research  applied  simulation  to 
experiment  with  a  small,  well-defined  element  of  the  larger  system.  Based  on  the  results 
of  these  experiments,  researchers  considered  how  their  result  could  be  applied  to  better 
understand  the  behavior  of  system  as  a  whole.  This  research  approaches  the  study  from  a 
macro  view,  directly  observing  the  simulated  interaction  of  major  system  elements. 

The  ARENA  interface  simplified  review  of  the  model  by  process  experts.  It 
allowed  direct  comparison  of  the  model  logic  to  a  conceptual  logic  flow  diagram 
developed  base  on  ARSC-provided  schedules  and  plans.  Similarly,  the  spreadsheet  view 
of  process  variables  within  ARENA  allows  researchers  to  easily  compare  programmed 
values  with  those  recorded  in  the  data  dictionary.  Given  that  simulation  is  well 
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established  as  a  tool  for  this  type  of  research,  ARENA  proved  to  be  a  good  choice  for  this 
study  because  its  interface  minimized  the  time  required  to  verify  and  validate  the  model. 

2.  How  well  does  the  ARENA  model  represent  the  effects  of  changes  in  labor  resources 
for  ARSC ’s  HH-60J  PDM  line  ? 

The  model  allows  researchers  to  consider  the  complex  interactions  between 
process  elements  with  variable  process  durations.  Because  ARSC’s  process  labor  data  is 
not  currently  formatted  for  direct  use  by  this  type  of  model,  the  exact  values  of  resulting 
process  responses  represent  only  a  predicted  trend.  However,  the  model  will 
accommodate  more  detailed  process  time  and  resource  requirement  data.  As  better  data 
become  available,  the  model’s  prediction  capability  will  improve. 

The  difference  in  variance  between  scenarios  suggests  that  additional  study  is 
warranted  to  better  understand  how  ARENA’S  resource  allocation  logic  affects  process 
time  in  the  simulation.  When  structures  shop  resources  are  not  required  for  beam 
replacement  work,  they  can  be  assigned  to  other  structures  work  tasks.  It  is  possible  for  a 
required  structures  resource  to  be  assigned  to  a  long-duration  task  before  an  entity  arrives 
for  beam  replacement.  In  this  case,  the  resource  is  not  released  from  the  current  task  to 
work  on  beam  replacement  until  the  task  is  completed,  causing  the  beam  replacement 
step  to  be  delayed.  This  ARENA- imposed  limitation  may  be  the  source  of  increased 
process  time  variation  in  scenario  two. 

3.  Will  an  increase  in  structures  shop  labor  likely  reduce  PDM  process  time  as 
compared  to  having  beam  replacement  work  done  by  a  dedicated  crew  of  contract 
workers  ? 
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Results  of  the  simulation  experiment  support  the  conclusion  that  shorter  process 
times  would  likely  be  achieved  by  increasing  structures  shop  manning.  In  the  simulation, 
this  manning  increase  cut  average  process  time  by  1.4  days  as  compared  to  having  the 
work  performed  by  a  contractor.  As  discussed  in  the  response  to  research  question  two, 
this  value  does  not  directly  represent  a  1.4  day  improvement  in  the  real  world  process. 
Rather,  it  simply  predicts  a  statistically  significant  reduction  in  average  process  time. 
Further  research  with  detailed  data  may  help  to  better  understand  the  magnitude  of  the 
expected  improvement. 

Practically  speaking,  analysts  would  likely  seek  process  improvements  that 
deliver  greater  reductions  in  process  time.  A  process  time  reduction  of  35  days  could 
decrease  the  fleet’ s  PDM  interval  from  five  years  to  four  years  with  a  constant  WIP  of 
five  aircraft.  Although  the  model  used  in  this  research  lacks  the  necessary  detail  to 
accurately  support  a  decision  to  reduce  PDM  intervals  based  on  an  untested  process 
change,  it  does  suggest  that  this  modeling  approach  has  the  potential  for  greater  insight 
into  the  effects  of  such  process  changes. 

4.  What  improvements  can  be  made  to  the  model  or  data  sources  to  develop  a  better 
ARENA  model  for  future  research  ? 

The  process  model  developed  for  this  research  is  a  starting  point  for  further 
research.  The  current  model  is  designed  to  study  the  impact  of  a  single  decision  variable. 
The  basic  process  structure  could  serve  as  the  basis  for  development  of  models  to 
consider  other  process  decisions.  These  might  include  modeling  the  effects  of  adding 
additional  process  steps,  constraining  facility  resources,  or  increasing  work  in  process. 
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Any  changes  to  the  model  or  its  intended  use  will  require  verification  and  validation  for 
the  intended  use. 

To  improve  the  predictive  capability  of  this  model  and  future  variants,  the 
following  data  should  be  updated  with  the  most  current  information  available: 

a.  Process  times  for  each  process  element  with  updated  probability  distributions. 

b.  Manpower  requirements  for  each  process  element,  categorized  by  work  shop 
when  multiple  shops  are  required  for  a  task. 

c.  Availability  of  workers  for  front  shop  work,  categorized  by  work  shop. 
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Appendix  A.  HH-60J  PDM  Logic  Flow  Diagrams 
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Appendix  A-l.  Main  Model  -  Logic  Flow 
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Appendix  A-2.  Disassembly  Sub-Model  -  Logic  Flow 
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Appendix  A-3.  Repair  Sub -Model  -  Logic  Flow 
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Appendix  A-4.  Interim  Paint  Sub-Model  -  Logic  Flow 
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Appendix  A-5.  Assembly  Sub-Model  -  Logic  Flow 
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Appendix  A-6.  Assembly  Sub-Model  -  Logic  Flow  (Continued) 
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Appendix  A-7.  Final  Paint  Sub-Model  -  Logic  Flow 
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Appendix  A-8.  Fuel  &  Ground  Runs  Sub-Model  -  Logic  Flow 
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Appendix  A-9.  Test  Flight  &  Ground  Runs  Sub-Model  -  Logic  Flow 
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Appendix  B.  HH-60J  PDM  ARENA  Logic  Diagrams 
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Appendix  B-l.  Main  Model  -  ARENA  Logic 
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Appendix  B-2.  Disassembly  Sub-Model  -  ARENA  Logic 
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Appendix  B-3.  Repair  Sub-Model  -  ARENA  Logic 
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Appendix  B-4.  Interim  Paint  Sub-Model  -  ARENA  Logic 
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Appendix  B-5.  Assembly  Sub-Model  -  ARENA  Logic 
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Appendix  B-6.  Paint  Sub-Model  -  ARENA  Logic 


o 


0 


61 


Appendix  B-7.  Fuel  &  Ground  Runs  Sub-Model  -  ARENA  Logic 
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Appendix  B-8.  Test  Flight  &  Ground  Runs  Sub-Model  -  ARENA  Logic 
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